This is the supplementary analytic output for the paper IJzerman et al., Social Thermoregulation: A Meta-analysis.

It reports detailed results for all models reported in the paper.

The analytic R script by which this html report was generated can be found on the project’s OSF page at: [LINK].

Brief information about the methods used in the analysis.


RMA results with model-based SEs k = number of studies; sqrt in “Variance components” = tau, the standard deviation of true effects; estimate in “Model results” = naive MA estimate

RVE SEs with Satterthwaite small-sample correction Estimate based on a multilevel RE model with constant sampling correlation model (CHE - correlated hierarchical effects - working model) (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Interpretation of naive-meta-analysis should be based on these estimates.

Prediction interval Shows the expected range of true effects in similar studies. As an approximation, in 95% of cases the true effect in a new published study can be expected to fall between PI LB and PI UB. Note that these are non-adjusted estimates. An unbiased newly conducted study will more likely fall in an interval centered around bias-adjusted estimate with a wider CI width.

Heterogeneity Tau can be interpreted as the total amount of heterogeneity in the true effects. I^2$ represents the ratio of true heterogeneity to total variance across the observed effect estimates. Estimates calculated by two approaches are reported. This is followed by separate estimates of between- and within-cluster heterogeneity and estimated intra-class correlation of underlying true effects.

Proportion of significant results What proportion of effects were statistically at the alpha level of .05.

ES-precision correlation Kendalls’s correlation between the ES and precision.

4/3PSM Applies a permutation-based, step-function 4-parameter selection model (one-tailed p-value steps = c(.025, .5, 1)). Falls back to 3-parameter selection model if at least one of the three p-value intervals contains less than 5 p-values. For this meta-analysis, we applied 3-parameter selection model by default as there were only 11 independent effects in the opposite direction overall (6%), causing the estimates to be unstable across iterations. pvalue = p-value testing H0 that the effect is zero. ciLB and ciUB are lower and upper bound of the CI. k = number of studies. steps = 3 means that the 4PSM was applied, 2 means that the 3PSM was applied.

PET-PEESE Estimated effect size of an infinitely precise study. Using 4/3PSM as the conditional estimator instead of PET (can be changed to PET). If the PET-PEESE estimate is in the opposite direction, the effect can be regarded nil. By default (can be changed to PET), the function employs a modified sample-size based estimator (see https://www.jepusto.com/pet-peese-performance/). It also uses the same RVE sandwich-type based estimator in a CHE (correlated hierarchical effects) working model with the identical random effects structure as the primary (naive) meta-analytic model.

We report results for both, PET and PEESE, with the first reported one being the primary (based on the conditional estimator).

WAAP-WLS The combined WAAP-WLS estimator (weighted average of the adequately powered - weighted least squares) tries to identify studies that are adequately powered to detect the meta-analytic effect. If there is less than two such studies, the method falls back to the WLS estimator (Stanley & Doucouliagos, 2015). If there are at least two adequately powered studies, WAAP returns a WLS estimate based on effects from only those studies.

type = 1: WAAP estimate, 2: WLS estimate. kAdequate = number of adequately powered studies

p-uniform P-uniform* is a selection model conceptually similar to p-curve. It makes use of the fact that p-values follow a uniform distribution at the true effect size while it includes also nonsignificant effect sizes. Permutation-based new version of p-uniform method, the so-called p-uniform* (van Aert, van Assen, 2021).

p-curve Permutation-based p-curve method. Output should be self-explanatory. For more info see p-curve.com

Power for detecting SESOI and bias-corrected parameter estimates Estimates of the statistical power for detecting a smallest effect sizes of interest equal to .20, .50, and .70 in SD units (Cohen’s d). A sort of a thought experiment, we also assumed that population true values equal the bias-corrected estimates (4/3PSM or PET-PEESE) and computed power for those.

Handling of dependencies in bias-correction methods To handle dependencies among the effects, the 4PSM, p-curve, p-uniform are implemented using a permutation-based procedure, randomly selecting only one focal effect (i.e., excluding those which were not coded as being focal) from a single study and iterating nIterations times. Lastly, the procedure selects the result with the median value of the ES estimate (4PSM, p-uniform) or median z-score of the full p-curve (p-curve).

Descriptives

Sample sizes

N of effects

## [1] 192

N of studies

## [1] 189

N of papers

## [1] 76

N of effects eligible just for evidential value test

## [1] 91

Median N across all the ES eligible for meta-analysis

## [1] 69

Meta-analysis results

Directionality of social thermoregulation effects

Source target directionality

## $`Model results`
## $`Model results`$test
##                                          Coef. Estimate     SE t-stat p-val (z)
## 1 factor(sourceTargetDirectionality_reconcil)0    0.465 0.0424  10.97    <0.001
## 2 factor(sourceTargetDirectionality_reconcil)1    0.421 0.0715   5.88    <0.001
##   Sig.
## 1  ***
## 2  ***
## 
## $`Model results`$CIs
##                                           Coef Estimate     SE d.f.
## 1 factor(sourceTargetDirectionality_reconcil)0    0.465 0.0424  Inf
## 2 factor(sourceTargetDirectionality_reconcil)1    0.421 0.0715  Inf
##   Lower 95% CI Upper 95% CI
## 1        0.382        0.548
## 2        0.281        0.561
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ 0.277      1     39.2 0.602

Moderation by prior experiences in relationships

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 19; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0177  0.1329      9     no         study 
## sigma^2.2  0.1215  0.3486     19     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 18) = 107.2633, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.5964  0.1208  4.9371  <.0001  0.3596  0.8332  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate   SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.596 0.12   4.96 6.46      0.00207   **
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate   SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.596 0.12 6.46        0.307        0.886
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.307     1.500 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3731153                    79.3790303 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    87.9600000                    10.0700000 
##  Within-cluster heterogeneity                           ICC 
##                    69.3000000                     0.1300000 
## 
## $`Proportion of significant results`
## [1] 0.7894737
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.2286703
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.340  0.265  1.284  0.199 -0.179  0.859  9.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##          0.177          0.431          0.411          0.694         -0.842 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          1.196          0.378          0.257          1.472          0.185 
##           ciLB           ciUB 
##         -0.229          0.986 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate std.error statistic     p.value  conf.low conf.high
## 1 WAAP-WLS   b0 0.5006211 0.1378544 0.1378544 0.006670333 0.1827282 0.8185139
##   type kAdequate
## 1    1         9
## 
## $`Publication bias`$`p-uniform*`
##        est       ciLB       ciUB     pvalue 
## 0.52602757 0.12599307 0.88075109 0.01115686 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 9 
## - Total number of p<0.05 studies included into the analysis: k = 7 (77.78%) 
## - Total number of studies with p<0.025: k = 6 (66.67%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.062 -7.277     0 -7.446     0
## Flatness test           0.905  5.095     1  6.936     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 98% (91%-99%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                      0.227 
##              Median power for detecting a SESOI of d = .50 
##                                                      0.856 
##              Median power for detecting a SESOI of d = .70 
##                                                      0.988 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                                                      0.188 
##             Median power for detecting 4/3PSM estimate.est 
##                                                      0.538

Forest plot and Contour-enhanced funnel plot

P-curve plot

Effect type

## [1] "The compensatory vs priming effects conceptualized by the actual direction of the effect as contrast vs. assimilation"

Compensatory

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 45; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0348  0.1866     35     no         study 
## sigma^2.2  0.0073  0.0855     45     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 44) = 132.7217, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3311  0.0446  7.4258  <.0001  0.2437  0.4185  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.331 0.0446   7.43 30.7       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.331 0.0446 30.7         0.24        0.422
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.096     0.758 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2052501                    71.6443532 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    83.2400000                    59.2000000 
##  Within-cluster heterogeneity                           ICC 
##                    12.4400000                     0.8300000 
## 
## $`Proportion of significant results`
## [1] 0.5106383
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.58586
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.241  0.069  3.505  0.000  0.106  0.375 35.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.214          0.067          3.200          0.003          0.078 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.351          0.038          0.117          0.326          0.747 
##           ciLB           ciUB 
##         -0.200          0.276 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate std.error statistic   p.value   conf.low conf.high
## 1 WAAP-WLS   b0 0.2408241 0.1099448 0.1099448 0.1162097 -0.1090692 0.5907174
##   type kAdequate
## 1    1         4
## 
## $`Publication bias`$`p-uniform*`
##           est          ciLB          ciUB        pvalue 
## 0.27407695862 0.14116727601 0.40194730157 0.00006139309 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 52 
## - Total number of p<0.05 studies included into the analysis: k = 35 (67.31%) 
## - Total number of studies with p<0.025: k = 23 (44.23%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.045 -6.934 0.000 -7.850     0
## Flatness test           0.282  3.093 0.999  9.726     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 69% (50.8%-81.9%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.385 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.986 
##                Median power for detecting a SESOI of d = .70 
##                                                        1.000 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.430 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.520

Priming

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 125; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0347  0.1864     84     no         study 
## sigma^2.2  0.0196  0.1399    125     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 124) = 315.2924, p-val < .0001
## 
## Model Results:
## 
## estimate      se     zval    pval   ci.lb   ci.ub 
##   0.4497  0.0339  13.2675  <.0001  0.3833  0.5162  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt     0.45 0.0339   13.3 73.9       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt     0.45 0.0339 73.9        0.382        0.517
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.019     0.918 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2330976                    60.9134712 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    76.8600000                    38.9600000 
##  Within-cluster heterogeneity                           ICC 
##                    21.9600000                     0.6400000 
## 
## $`Proportion of significant results`
## [1] 0.6136364
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5434097
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.262  0.052  5.047  0.000  0.160  0.363 82.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.204          0.056          3.656          0.000          0.093 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.315         -0.049          0.060         -0.810          0.421 
##           ciLB           ciUB 
##         -0.169          0.071 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term   estimate  std.error  statistic   p.value    conf.low
## 1 WAAP-WLS   b0 0.06033676 0.03493944 0.03493944 0.1349326 -0.02515696
##   conf.high type kAdequate
## 1 0.1458305    1         7
## 
## $`Publication bias`$`p-uniform*`
##              est             ciLB             ciUB           pvalue 
## 0.29093860463409 0.18771549135640 0.39108082217172 0.00000004950953 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 119 
## - Total number of p<0.05 studies included into the analysis: k = 81 (68.07%) 
## - Total number of studies with p<0.025: k = 50 (42.02%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.022 -5.674  0.00 -7.395     0
## Flatness test           0.038 -0.177  0.43  9.756     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 32% (19.8%-45.5%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.202 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.801 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.976 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.209 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.313

Table 1

##              k   g [95% CI]        SE   Tau  I^2 3PSM est [95% CI] 3PSM.pvalue
## Compensatory 47  0.33 [0.24, 0.42] 0.04 0.21 72% 0.24 [0.11, 0.38] 0          
## Priming      132 0.45 [0.38, 0.52] 0.03 0.23 61% 0.26 [0.16, 0.36] 0          
##              PET-PEESE est [95% CI] PET-PEESE.pvalue
## Compensatory 0.21 [0.08, 0.35]      0.003           
## Priming      0.2 [0.09, 0.32]       0

Comparison of effect types

Model without covariates

## $`Model results`
## $`Model results`$test
##                        Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(effectCompPriming)1    0.345 0.0461   7.48    <0.001  ***
## 2 factor(effectCompPriming)2    0.443 0.0324  13.65    <0.001  ***
## 
## $`Model results`$CIs
##                         Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(effectCompPriming)1    0.345 0.0461  Inf        0.254        0.435
## 2 factor(effectCompPriming)2    0.443 0.0324  Inf        0.379        0.506
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  2.81      1     47.5   0.1

Model with covariates

Controlling for design-related factors that are prognostic w.r.t. the effect sizes (i.e., might vary across moderator categories), namely rct, published, sourceTargetDirectionality, and studentSample.

## $`Model results`
## $`Model results`$test
##                                 Coef. Estimate     SE t-stat p-val (z) Sig.
## 1          factor(effectCompPriming)1   0.0932 0.0721  1.293   0.19600     
## 2          factor(effectCompPriming)2   0.1769 0.0898  1.969   0.04896    *
## 3                                 rct   0.1463 0.0564  2.595   0.00945   **
## 4                           published   0.1285 0.0630  2.040   0.04138    *
## 5 sourceTargetDirectionality_reconcil  -0.0305 0.0681 -0.448   0.65428     
## 6                       studentSample   0.0789 0.0684  1.153   0.24873     
## 
## $`Model results`$CIs
##                                  Coef Estimate     SE d.f. Lower 95% CI
## 1          factor(effectCompPriming)1   0.0932 0.0721  Inf    -0.048070
## 2          factor(effectCompPriming)2   0.1769 0.0898  Inf     0.000805
## 3                                 rct   0.1463 0.0564  Inf     0.035802
## 4                           published   0.1285 0.0630  Inf     0.005026
## 5 sourceTargetDirectionality_reconcil  -0.0305 0.0681  Inf    -0.163877
## 6                       studentSample   0.0789 0.0684  Inf    -0.055188
##   Upper 95% CI
## 1        0.234
## 2        0.353
## 3        0.257
## 4        0.252
## 5        0.103
## 6        0.213
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  2.13      1     26.1 0.156

Plots

Forest plots

Contour-enhanced funnel plots

p-curve plots

PET-PEESE plots

top = Compensatory, bottom = Priming

Using the sqrt(2/n) and 2/n terms instead of SE and var for PET and PEESE, respectively since modified sample-size based estimator was implemented (see https://www.jepusto.com/pet-peese-performance/).

Mood

Results

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 17; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000     11     no         study 
## sigma^2.2  0.0000  0.0000     17     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 16) = 7.1492, p-val = 0.9703
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.2315  0.0698  3.3145  0.0009  0.0946  0.3684  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.231 0.0448   5.17  8.9       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.231 0.0448  8.9         0.13        0.333
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.131     0.332 
## 
## $Heterogeneity
##                           Tau                           I^2 
##              0.00000295647900              0.00000001354476 
##                 Jackson's I^2 Between-cluster heterogeneity 
##              0.00000000000000              0.00000000000000 
##  Within-cluster heterogeneity                           ICC 
##              0.00000000000000              1.00000000000000 
## 
## $`Proportion of significant results`
## [1] 0.1176471
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Forest plot and Contour-enhanced funnel plot

Overall effect results

Results

Number of iterations run equal to 3 for p-curve and 3 for all other bias correction functions.

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 175; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0438  0.2092    116     no         study 
## sigma^2.2  0.0357  0.1888    175     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 174) = 677.7377, p-val < .0001
## 
## Model Results:
## 
## estimate      se     zval    pval   ci.lb   ci.ub 
##   0.4085  0.0315  12.9864  <.0001  0.3469  0.4702  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.409 0.0315     13  105       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.409 0.0315  105        0.346        0.471
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.153     0.970 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2818135                    74.9082576 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    85.1400000                    41.2700000 
##  Within-cluster heterogeneity                           ICC 
##                    33.6400000                     0.5500000 
## 
## $`Proportion of significant results`
## [1] 0.5923913
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5796001
## 
## $`Publication bias`$`4/3PSM`
##     est      se  zvalue  pvalue    ciLB    ciUB       k   steps 
##   0.193   0.047   4.105   0.000   0.101   0.285 115.000   2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.159          0.053          3.008          0.003          0.054 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.264         -0.138          0.072         -1.915          0.058 
##           ciLB           ciUB 
##         -0.280          0.005 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic   p.value   conf.low conf.high
## 1 WAAP-WLS   b0 0.1493071 0.09129957 0.09129957 0.2436033 -0.2435232 0.5421375
##   type kAdequate
## 1    1         3
## 
## $`Publication bias`$`p-uniform*`
##            est           ciLB           ciUB         pvalue 
## 0.217868891545 0.122307236949 0.312420179661 0.000007962177 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 164 
## - Total number of p<0.05 studies included into the analysis: k = 113 (68.9%) 
## - Total number of studies with p<0.025: k = 70 (42.68%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull   zHalf pHalf
## Right-skewness test     0.007 -8.517 0.000 -10.904     0
## Flatness test           0.019  1.492 0.932  13.228     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 44% (32.3%-54.8%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.215 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.830 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.983 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.153 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.203

Forest plot

Contour-enhanced funnel plot

p-curve plot

Methods

Moderator analysis

## $`Model results`
## $`Model results`$test
##                                     Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 methodPhysical temperature manipulation    0.464 0.0629   7.37   < 0.001  ***
## 2   methodVisual/verbal temperature prime    0.485 0.0589   8.23   < 0.001  ***
## 3               methodOutside temperature    0.379 0.1512   2.51   0.01223    *
## 4        methodTemperature estimate as DV    0.477 0.0703   6.79   < 0.001  ***
## 5  methodSubjective warmth judgment as DV    0.299 0.1149   2.61   0.00915   **
## 
## $`Model results`$CIs
##                                      Coef Estimate     SE d.f. Lower 95% CI
## 1 methodPhysical temperature manipulation    0.464 0.0629  Inf       0.3406
## 2   methodVisual/verbal temperature prime    0.485 0.0589  Inf       0.3693
## 3               methodOutside temperature    0.379 0.1512  Inf       0.0825
## 4        methodTemperature estimate as DV    0.477 0.0703  Inf       0.3396
## 5  methodSubjective warmth judgment as DV    0.299 0.1149  Inf       0.0743
##   Upper 95% CI
## 1        0.587
## 2        0.600
## 3        0.675
## 4        0.615
## 5        0.525
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ 0.682      4     17.5 0.614

Subgroup analysis

Leaving out the Core temperature measurement and Skin temperature measurement, since k is too low.

Meta-analysis results

Brief results

##                                   k  g [95% CI]         SE   Tau  I^2
## Physical.temperature.manipulation 83 0.48 [0.36, 0.59]  0.06 0.34 69%
## Visual.verbal.temperature.prime   23 0.44 [0.35, 0.53]  0.04 0.15 38%
## Outside.temperature               13 0.16 [-0.02, 0.34] 0.06 0.1  44%
## Temperature.estimate.as.DV        25 0.39 [0.27, 0.52]  0.06 0.19 60%
## Subjective.warmth.judgment.as.DV  14 0.28 [0.03, 0.53]  0.11 0.37 93%

Physical temperature manipulation

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 82; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.1003  0.3167     51     no         study 
## sigma^2.2  0.0153  0.1238     82     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 81) = 339.5264, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4779  0.0575  8.3075  <.0001  0.3652  0.5907  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.478 0.0576    8.3 48.2       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.478 0.0576 48.2        0.362        0.594
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.215     1.171 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3400426                    69.0555528 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    80.9300000                    59.9000000 
##  Within-cluster heterogeneity                           ICC 
##                     9.1500000                     0.8700000 
## 
## $`Proportion of significant results`
## [1] 0.6746988
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Visual/verbal temperature prime

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 23; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000     14     no         study 
## sigma^2.2  0.0232  0.1524     23     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 22) = 38.5260, p-val = 0.0160
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4398  0.0600  7.3355  <.0001  0.3223  0.5573  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt     0.44 0.0411   10.7 11.4       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt     0.44 0.0411 11.4         0.35         0.53
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.099     0.781 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1523991                    37.8013636 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    13.6800000                     0.0000000 
##  Within-cluster heterogeneity                           ICC 
##                    37.8000000                     0.0000000 
## 
## $`Proportion of significant results`
## [1] 0.6956522
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Outside temperature

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 8; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0077  0.0875      6     no         study 
## sigma^2.2  0.0028  0.0525      8     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 7) = 14.2050, p-val = 0.0477
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.1607  0.0642  2.5034  0.0123  0.0349  0.2864  * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.161 0.0634   2.54 3.63       0.0705    .
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.161 0.0634 3.63      -0.0225        0.344
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.147     0.468 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1020439                    44.1844740 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    56.9100000                    32.5100000 
##  Within-cluster heterogeneity                           ICC 
##                    11.6700000                     0.7400000 
## 
## $`Proportion of significant results`
## [1] 0.1538462
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Temperature estimate as DV

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 23; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0378  0.1945     21     no         study 
## sigma^2.2  0.0000  0.0000     23     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 22) = 49.5679, p-val = 0.0007
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3931  0.0599  6.5631  <.0001  0.2757  0.5104  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.393 0.0601   6.54   17       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.393 0.0601   17        0.266         0.52
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.031     0.817 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1945094                    59.9035594 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    71.0500000                    59.9000000 
##  Within-cluster heterogeneity                           ICC 
##                     0.0000000                     1.0000000 
## 
## $`Proportion of significant results`
## [1] 0.72
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Subjective warmth judgment as DV

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 13; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.1253  0.3539     12     no         study 
## sigma^2.2  0.0089  0.0942     13     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 12) = 69.2488, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.2810  0.1153  2.4362  0.0148  0.0549  0.5070  * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate    SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.281 0.115   2.45 10.7        0.033    *
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.281 0.115 10.7       0.0274        0.535
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.564     1.126 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3662546                    92.6858591 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    95.7600000                    86.5600000 
##  Within-cluster heterogeneity                           ICC 
##                     6.1300000                     0.9300000 
## 
## $`Proportion of significant results`
## [1] 0.5714286
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Bias correction results

Physical temperature manipulation

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5226101
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.152  0.093  1.644  0.100 -0.029  0.334 50.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.254          0.225         -1.128          0.265         -0.705 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.198          0.190          0.133          1.424          0.161 
##           ciLB           ciUB 
##         -0.078          0.457 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term   estimate std.error statistic   p.value  conf.low conf.high
## 1 WAAP-WLS   b0 -0.4364618  0.251926  0.251926 0.2253231 -1.520412 0.6474885
##   type kAdequate
## 1    1         3
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
##  0.06337261 -0.13821013  0.26449936  0.53861375 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 81 
## - Total number of p<0.05 studies included into the analysis: k = 60 (74.07%) 
## - Total number of studies with p<0.025: k = 38 (46.91%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.026 -7.180 0.000 -8.491     0
## Flatness test           0.109  2.094 0.982 10.242     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 53% (37.5%-66.6%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                     "0.19" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.768" 
##              Median power for detecting a SESOI of d = .70 
##                                                    "0.965" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                     "0.13"

Visual/verbal temperature prime

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.3287924
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.263  0.123  2.143  0.032  0.022  0.504 14.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.292          0.110          2.662          0.021          0.053 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.532          0.182          0.207          0.878          0.397 
##           ciLB           ciUB 
##         -0.270          0.634 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic     p.value  conf.low conf.high
## 1 WAAP-WLS   b0 0.3037822 0.03003057 0.03003057 0.009631472 0.1745711 0.4329934
##   type kAdequate
## 1    1         3
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
## 0.323360818 0.112586407 0.505288265 0.003654316 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 23 
## - Total number of p<0.05 studies included into the analysis: k = 19 (82.61%) 
## - Total number of studies with p<0.025: k = 14 (60.87%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.032 -4.527 0.000 -4.793     0
## Flatness test           0.670  1.450 0.926  5.666     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 56% (30.3%-77.3%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.202 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.801 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.976 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.374 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.315

Temperature estimate as DV

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.7500059
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.022  0.055  0.392  0.695 -0.087  0.130 21.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.086          0.091         -0.942          0.358         -0.277 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.105          0.108          0.062          1.742          0.098 
##           ciLB           ciUB 
##         -0.022          0.238 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term   estimate std.error statistic   p.value  conf.low conf.high
## 1 WAAP-WLS   b0 0.04964282 0.1242567 0.1242567 0.7580266 -1.529189  1.628474
##   type kAdequate
## 1    1         2
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
##  0.11862207 -0.05951828  0.28513618  0.18952115 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 24 
## - Total number of p<0.05 studies included into the analysis: k = 18 (75%) 
## - Total number of studies with p<0.025: k = 10 (41.67%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.407 -1.013 0.156 -2.116 0.017
## Flatness test           0.112 -1.538 0.062  3.648 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 11% (5%-35.3%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                    "0.202" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.801" 
##              Median power for detecting a SESOI of d = .70 
##                                                    "0.976" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                    "0.052"

Contour-enhanced funnel plots

Forest plots

Categories

Leaving out the Robotics and Neural Mechanisms, since k is too low

Results for different categories

Meta-analysis results

Brief results

##                          k  g [95% CI]        SE   Tau  I^2
## Emotion                  24 0.39 [0.31, 0.46] 0.03 0.15 54%
## Interpersonal            76 0.36 [0.26, 0.46] 0.05 0.31 78%
## Person.perception        39 0.41 [0.23, 0.58] 0.08 0.33 81%
## Group.processes          12 0.62 [0.38, 0.85] 0.09 0    0% 
## Moral.judgment           6  0.49 [-0.12, 1.1] 0.11 0.03 2% 
## Self.regulation          26 0.32 [0.17, 0.47] 0.07 0.27 76%
## Cognitive.processes      36 0.56 [0.46, 0.66] 0.05 0.12 20%
## Economic.decision.making 43 0.44 [0.28, 0.59] 0.07 0.31 70%

Emotion

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 23; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000     19     no         study 
## sigma^2.2  0.0233  0.1526     23     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 22) = 44.6519, p-val = 0.0029
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3859  0.0506  7.6308  <.0001  0.2868  0.4850  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.386 0.0343   11.2 15.3       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.386 0.0343 15.3        0.313        0.459
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.057     0.715 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1526369                    54.0804386 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    65.7500000                     0.0000000 
##  Within-cluster heterogeneity                           ICC 
##                    54.0800000                     0.0000000 
## 
## $`Proportion of significant results`
## [1] 0.7083333
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Interpersonal

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 75; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0503  0.2242     56     no         study 
## sigma^2.2  0.0488  0.2208     75     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 74) = 364.3079, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3594  0.0498  7.2141  <.0001  0.2617  0.4570  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.359 0.0499   7.21 51.7       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.359 0.0499 51.7        0.259        0.459
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.279     0.998 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3146483                    77.6948925 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    85.2200000                    39.4400000 
##  Within-cluster heterogeneity                           ICC 
##                    38.2600000                     0.5100000 
## 
## $`Proportion of significant results`
## [1] 0.5789474
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Person perception

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 36; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0765  0.2766     21     no         study 
## sigma^2.2  0.0337  0.1835     36     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 35) = 133.8868, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4075  0.0829  4.9175  <.0001  0.2451  0.5698  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.407 0.0828   4.92 18.3       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.407 0.0828 18.3        0.234        0.581
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.306     1.121 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.3319022                    81.0462124 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    90.0500000                    56.2800000 
##  Within-cluster heterogeneity                           ICC 
##                    24.7700000                     0.6900000 
## 
## $`Proportion of significant results`
## [1] 0.4358974
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Group processes

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 11; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000      7     no         study 
## sigma^2.2  0.0000  0.0000     11     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 10) = 8.8815, p-val = 0.5434
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.6152  0.1079  5.7024  <.0001  0.4037  0.8266  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate    SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.615 0.094   6.55  5.3       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.615 0.094  5.3        0.378        0.853
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.386     0.845 
## 
## $Heterogeneity
##                           Tau                           I^2 
##              0.00000462993551              0.00000002010181 
##                 Jackson's I^2 Between-cluster heterogeneity 
##              0.00000000000000              0.00000000000000 
##  Within-cluster heterogeneity                           ICC 
##              0.00000000000000              0.83000000000000 
## 
## $`Proportion of significant results`
## [1] 0.8333333
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Moral judgment

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 6; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000      3     no         study 
## sigma^2.2  0.0011  0.0329      6     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 5) = 4.6596, p-val = 0.4588
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4878  0.1076  4.5335  <.0001  0.2769  0.6987  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate    SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.488 0.111   4.41 1.61       0.0701    .
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.488 0.111 1.61        -0.12          1.1
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.027     0.948 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                    0.03292223                    2.34692161 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                   25.18000000                    0.00000000 
##  Within-cluster heterogeneity                           ICC 
##                    2.35000000                    0.00000000 
## 
## $`Proportion of significant results`
## [1] NA
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Self-regulation

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 24; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0312  0.1765     21     no         study 
## sigma^2.2  0.0436  0.2089     24     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 23) = 90.2528, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.3242  0.0716  4.5298  <.0001  0.1839  0.4645  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.324 0.0714   4.54 18.7       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.324 0.0714 18.7        0.175        0.474
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.265     0.914 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2734749                    76.4112635 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    84.2000000                    31.8400000 
##  Within-cluster heterogeneity                           ICC 
##                    44.5800000                     0.4200000 
## 
## $`Proportion of significant results`
## [1] 0.5384615
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Cognitive processes

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 35; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0000  0.0000     29     no         study 
## sigma^2.2  0.0143  0.1194     35     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 34) = 40.8129, p-val = 0.1959
## 
## Model Results:
## 
## estimate      se     zval    pval   ci.lb   ci.ub 
##   0.5566  0.0485  11.4785  <.0001  0.4616  0.6517  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.557 0.0478   11.6 23.2       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.557 0.0478 23.2        0.458        0.655
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##     0.293     0.820 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1194099                    19.7930556 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    26.6100000                     0.0000000 
##  Within-cluster heterogeneity                           ICC 
##                    19.7900000                     0.0000000 
## 
## $`Proportion of significant results`
## [1] 0.75
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Economic decision-making

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 38; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0671  0.2590     25     no         study 
## sigma^2.2  0.0306  0.1750     38     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 37) = 117.7860, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.4361  0.0743  5.8697  <.0001  0.2905  0.5818  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.436 0.0743   5.87 22.7       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.436 0.0743 22.7        0.282         0.59
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.227     1.099 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                      0.312574                     70.182750 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                     81.650000                     48.180000 
##  Within-cluster heterogeneity                           ICC 
##                     22.000000                      0.690000 
## 
## $`Proportion of significant results`
## [1] 0.5581395
## 
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"

Bias correction results

Emotion

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.2669407
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.261  0.028  9.249  0.000  0.206  0.316 19.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.353          0.050          7.110          0.000          0.249 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.458          0.301          0.064          4.724          0.000 
##           ciLB           ciUB 
##          0.167          0.436 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic           p.value  conf.low
## 1 WAAP-WLS   b0 0.3105771 0.02925429 0.02925429 0.000000000402088 0.2499074
##   conf.high type kAdequate
## 1 0.3712468    2         1
## 
## $`Publication bias`$`p-uniform*`
## numeric(0)
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 32 
## - Total number of p<0.05 studies included into the analysis: k = 25 (78.12%) 
## - Total number of studies with p<0.025: k = 14 (43.75%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.345 -4.808  0.00 -7.055     0
## Flatness test           0.073  1.473  0.93  7.490     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 55% (30.9%-74.5%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.331 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.968 
##                Median power for detecting a SESOI of d = .70 
##                                                        1.000 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.767 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.511

Interpersonal

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5905207
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.095  0.078  1.220  0.222 -0.057  0.247 56.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.215          0.107         -2.000          0.051         -0.430 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.000          0.098          0.077          1.276          0.207 
##           ciLB           ciUB 
##         -0.056          0.252 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic     p.value   conf.low
## 1 WAAP-WLS   b0 0.1388678 0.04068096 0.04068096 0.001043176 0.05780919
##   conf.high type kAdequate
## 1 0.2199264    2         1
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
##  0.03429508 -0.11339811  0.18536407  0.65236324 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 87 
## - Total number of p<0.05 studies included into the analysis: k = 55 (63.22%) 
## - Total number of studies with p<0.025: k = 33 (37.93%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.089 -5.150 0.000 -7.024     0
## Flatness test           0.045  0.502 0.692  9.039     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 38% (22.7%-54.8%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                    "0.205" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.808" 
##              Median power for detecting a SESOI of d = .70 
##                                                    "0.977" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                    "0.084"

Person perception

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6107588
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.491  0.139  3.539  0.000  0.219  0.763 21.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.060          0.050          1.181          0.252         -0.046 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.165         -0.236          0.064         -3.710          0.001 
##           ciLB           ciUB 
##         -0.369         -0.103 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic      p.value   conf.low
## 1 WAAP-WLS   b0 0.1963766 0.05084897 0.05084897 0.0004645318 0.09314768
##   conf.high type kAdequate
## 1 0.2996055    2         0
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
## 0.379250558 0.101527625 0.656972822 0.007021934 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 27 
## - Total number of p<0.05 studies included into the analysis: k = 16 (59.26%) 
## - Total number of studies with p<0.025: k = 11 (40.74%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.105 -5.983 0.000 -7.551     0
## Flatness test           0.500  2.997 0.999  7.140     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 78% (55.7%-90.8%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.253 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.899 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.995 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.067 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.888

Self-regulation

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6350534
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.068  0.114  0.600  0.548 -0.155  0.292 21.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.318          0.101         -3.148          0.005         -0.530 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##         -0.107         -0.003          0.056         -0.057          0.955 
##           ciLB           ciUB 
##         -0.120          0.114 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term estimate  std.error  statistic    p.value   conf.low conf.high
## 1 WAAP-WLS   b0   0.1496 0.05771728 0.05771728 0.01630102 0.03020272 0.2689973
##   type kAdequate
## 1    2         1
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
##  0.13278230 -0.06309732  0.33006479  0.18636257 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 35 
## - Total number of p<0.05 studies included into the analysis: k = 22 (62.86%) 
## - Total number of studies with p<0.025: k = 14 (40%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.143 -2.328 0.010 -1.654 0.049
## Flatness test           0.276 -0.648 0.259  4.128 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 24% (7.5%-49.5%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                    "0.396" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.989" 
##              Median power for detecting a SESOI of d = .70 
##                                                        "1" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                    "0.089"

Cognitive processes

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.3800724
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.351  0.097  3.619  0.000  0.161  0.541 29.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.405          0.083          4.859          0.000          0.234 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.576          0.218          0.146          1.490          0.148 
##           ciLB           ciUB 
##         -0.082          0.519 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate std.error statistic    p.value  conf.low conf.high
## 1 WAAP-WLS   b0 0.4210579 0.1142019 0.1142019 0.02107484 0.1039825 0.7381332
##   type kAdequate
## 1    1         5
## 
## $`Publication bias`$`p-uniform*`
##        est       ciLB       ciUB     pvalue 
## 0.27581401 0.05196072 0.47926301 0.01663041 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 51 
## - Total number of p<0.05 studies included into the analysis: k = 37 (72.55%) 
## - Total number of studies with p<0.025: k = 24 (47.06%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.049 -5.577 0.000 -6.570     0
## Flatness test           0.238  1.483 0.931  7.809     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 51% (31.5%-68.3%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.197 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.789 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.972 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.609 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.492

Economic decision-making

## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6508614
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.231  0.126  1.841  0.066 -0.015  0.478 24.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.241          0.134          1.792          0.086         -0.037 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.519         -0.135          0.257         -0.526          0.604 
##           ciLB           ciUB 
##         -0.668          0.397 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic       p.value  conf.low
## 1 WAAP-WLS   b0 0.3017689 0.05936583 0.05936583 0.00001090482 0.1814823
##   conf.high type kAdequate
## 1 0.4220555    2         0
## 
## $`Publication bias`$`p-uniform*`
##         est        ciLB        ciUB      pvalue 
##  0.21630770 -0.03949487  0.46801677  0.09774954 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 37 
## - Total number of p<0.05 studies included into the analysis: k = 28 (75.68%) 
## - Total number of studies with p<0.025: k = 16 (43.24%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.286 -2.609 0.005 -2.998 0.001
## Flatness test           0.075 -0.761 0.223  4.604 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 23% (8.5%-46%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.202 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.801 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.976 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.272 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.254

Contour enhanced funnel plots

Forest plots

Moderator/sensitivity analyses

The below reported meta-regressions are all implemented as a multivariate RVE-based models using the CHE working model (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Testing of contrasts is carried out using a robust Wald-type test testing the equality of estimates across levels of the moderator.

Moderation by citations and IF

Overall effect moderated by citations and IF

## $test
##                              Coef. Estimate     SE t-stat p-val (z) Sig.
## 1                          intrcpt   0.4209 0.0307  13.70    <0.001  ***
## 2           scale(publicationYear)  -0.0334 0.0293  -1.14    0.2535     
## 3      scale(citationsGSMarch2016)   0.0626 0.0309   2.03    0.0424    *
## 4 scale(h5indexGSJournalMarch2016)  -0.0880 0.0403  -2.18    0.0290    *
## 
## $CIs
##                               Coef Estimate     SE d.f. Lower 95% CI
## 1                          intrcpt   0.4209 0.0307  Inf      0.36071
## 2           scale(publicationYear)  -0.0334 0.0293  Inf     -0.09081
## 3      scale(citationsGSMarch2016)   0.0626 0.0309  Inf      0.00214
## 4 scale(h5indexGSJournalMarch2016)  -0.0880 0.0403  Inf     -0.16688
##   Upper 95% CI
## 1      0.48116
## 2      0.02395
## 3      0.12315
## 4     -0.00903

Moderation by lattitude

Overall effect moderated by lattitude

## $test
##                       Coef. Estimate     SE t-stat p-val (z) Sig.
## 1                   intrcpt   0.4584 0.0367 12.482    <0.001  ***
## 2 scale(latitudeUniversity)   0.0267 0.0362  0.737     0.461     
## 
## $CIs
##                        Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1                   intrcpt   0.4584 0.0367  Inf       0.3864       0.5304
## 2 scale(latitudeUniversity)   0.0267 0.0362  Inf      -0.0442       0.0975

Compensatory effects moderated by lattitude

## $test
##                       Coef. Estimate     SE t-stat p-val (z) Sig.
## 1                   intrcpt   0.3051 0.0516  5.913    <0.001  ***
## 2 scale(latitudeUniversity)  -0.0323 0.0528 -0.612     0.541     
## 
## $CIs
##                        Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1                   intrcpt   0.3051 0.0516  Inf        0.204       0.4063
## 2 scale(latitudeUniversity)  -0.0323 0.0528  Inf       -0.136       0.0712

Priming effects moderated by lattitude

## $test
##                       Coef. Estimate    SE t-stat p-val (z) Sig.
## 1                   intrcpt   0.5120 0.037  13.86    <0.001  ***
## 2 scale(latitudeUniversity)   0.0451 0.037   1.22     0.223     
## 
## $CIs
##                        Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1                   intrcpt   0.5120 0.037  Inf       0.4396        0.584
## 2 scale(latitudeUniversity)   0.0451 0.037  Inf      -0.0274        0.118

Mood effects moderated by lattitude

## $test
##                       Coef. Estimate     SE t-stat p-val (z) Sig.
## 1                   intrcpt    0.232 0.0463  5.017    <0.001  ***
## 2 scale(latitudeUniversity)   -0.016 0.0353 -0.451     0.652     
## 
## $CIs
##                        Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1                   intrcpt    0.232 0.0463  Inf       0.1417       0.3233
## 2 scale(latitudeUniversity)   -0.016 0.0353  Inf      -0.0852       0.0533

Moderation by gender

Overall effect moderated by gender ratio

## $test
##               Coef. Estimate     SE t-stat p-val (z) Sig.
## 1           intrcpt    0.427 0.0338  12.63   < 0.001  ***
## 2 scale(percFemale)    0.103 0.0388   2.64   0.00821   **
## 
## $CIs
##                Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1           intrcpt    0.427 0.0338  Inf       0.3609        0.494
## 2 scale(percFemale)    0.103 0.0388  Inf       0.0266        0.179

Compensatory effects moderated by gender ratio

## $test
##               Coef. Estimate     SE t-stat p-val (z) Sig.
## 1           intrcpt   0.3062 0.0494  6.204    <0.001  ***
## 2 scale(percFemale)  -0.0249 0.0568 -0.438     0.661     
## 
## $CIs
##                Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1           intrcpt   0.3062 0.0494  Inf        0.209       0.4030
## 2 scale(percFemale)  -0.0249 0.0568  Inf       -0.136       0.0864

Priming effects moderated by gender ratio

## $test
##               Coef. Estimate     SE t-stat p-val (z) Sig.
## 1           intrcpt    0.467 0.0352   13.3    <0.001  ***
## 2 scale(percFemale)    0.148 0.0422    3.5    <0.001  ***
## 
## $CIs
##                Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1           intrcpt    0.467 0.0352  Inf        0.398        0.536
## 2 scale(percFemale)    0.148 0.0422  Inf        0.065        0.231

Published status

## $`Model results`
## $`Model results`$test
##                Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(published)0    0.312 0.0773   4.03    <0.001  ***
## 2 factor(published)1    0.432 0.0347  12.47    <0.001  ***
## 
## $`Model results`$CIs
##                 Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(published)0    0.312 0.0773  Inf        0.160        0.463
## 2 factor(published)1    0.432 0.0347  Inf        0.364        0.500
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  2.05      1     24.3 0.164

Comparing randomized and non-randomized designs

## $`Model results`
## $`Model results`$test
##          Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(rct)0    0.260 0.0543   4.79    <0.001  ***
## 2 factor(rct)1    0.447 0.0360  12.40    <0.001  ***
## 
## $`Model results`$CIs
##           Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(rct)0    0.260 0.0543  Inf        0.154        0.367
## 2 factor(rct)1    0.447 0.0360  Inf        0.376        0.517
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom   p_val sig
##   HTZ  8.72      1     34.7 0.00562  **

Subgroups of observational and randomized effects

##                k   g [95% CI]        SE   Tau  I^2 3PSM est [95% CI] 
## Non.randomized 34  0.26 [0.14, 0.37] 0.06 0.22 80% 0.05 [-0.06, 0.16]
## Randomized     148 0.45 [0.38, 0.52] 0.04 0.29 70% 0.19 [0.07, 0.31] 
##                3PSM.pvalue PET-PEESE est [95% CI] PET-PEESE.pvalue
## Non.randomized 0.33        -0.1 [-0.31, 0.12]     0.358           
## Randomized     0.002       0.19 [0.04, 0.33]      0.011

Non-randomized

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 29; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0387  0.1968     24     no         study 
## sigma^2.2  0.0116  0.1078     29     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 28) = 102.6511, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub 
##   0.2554  0.0567  4.5059  <.0001  0.1443  0.3665  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.255 0.0564   4.52 21.1       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.255 0.0564 21.1        0.138        0.373
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.223     0.734 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2243704                    79.6290061 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    89.8100000                    61.2600000 
##  Within-cluster heterogeneity                           ICC 
##                    18.3700000                     0.7700000 
## 
## $`Proportion of significant results`
## [1] 0.3823529
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.4714771
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.054  0.056  0.974  0.330 -0.055  0.164 23.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.097          0.104         -0.938          0.358         -0.312 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.118          0.094          0.061          1.536          0.139 
##           ciLB           ciUB 
##         -0.033          0.222 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term  estimate  std.error  statistic   p.value   conf.low conf.high
## 1 WAAP-WLS   b0 0.1493071 0.09129957 0.09129957 0.2436033 -0.2435232 0.5421375
##   type kAdequate
## 1    1         3
## 
## $`Publication bias`$`p-uniform*`
##          est         ciLB         ciUB       pvalue 
##  0.120707963 -0.004595503  0.249183123  0.059241275 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 23 
## - Total number of p<0.05 studies included into the analysis: k = 11 (47.83%) 
## - Total number of studies with p<0.025: k = 7 (30.43%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.274 -3.271 0.001 -3.501     0
## Flatness test           0.389  1.111 0.867  4.880     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 58% (22.5%-83.7%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##              Median power for detecting a SESOI of d = .20 
##                                                    "0.367" 
##              Median power for detecting a SESOI of d = .50 
##                                                    "0.982" 
##              Median power for detecting a SESOI of d = .70 
##                                                        "1" 
## Median power for detecting PET-PEESE estimate.PET estimate 
##                    "ES estimate in the opposite direction" 
##             Median power for detecting 4/3PSM estimate.est 
##                                                    "0.072"

Randomized

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 144; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0464  0.2155     92     no         study 
## sigma^2.2  0.0364  0.1907    144     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 143) = 559.4052, p-val < .0001
## 
## Model Results:
## 
## estimate      se     zval    pval   ci.lb   ci.ub 
##   0.4521  0.0364  12.4333  <.0001  0.3809  0.5234  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt    0.452 0.0364   12.4 82.9       <0.001  ***
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt    0.452 0.0364 82.9         0.38        0.525
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.124     1.028 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.2877887                    69.8305082 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    80.4600000                    39.1500000 
##  Within-cluster heterogeneity                           ICC 
##                    30.6800000                     0.5600000 
## 
## $`Proportion of significant results`
## [1] 0.6351351
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5386684
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
##  0.190  0.061  3.095  0.002  0.070  0.310 92.000  2.000 
## 
## $`Publication bias`$`PET-PEESE`
## PEESE estimate             se         zvalue         pvalue           ciLB 
##          0.186          0.072          2.586          0.011          0.043 
##           ciUB   PET estimate             se         zvalue         pvalue 
##          0.329         -0.145          0.096         -1.511          0.134 
##           ciLB           ciUB 
##         -0.336          0.046 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term    estimate std.error statistic   p.value   conf.low conf.high
## 1 WAAP-WLS   b0 -0.02180643 0.1306439 0.1306439 0.8729201 -0.3414805 0.2978676
##   type kAdequate
## 1    1         7
## 
## $`Publication bias`$`p-uniform*`
##        est       ciLB       ciUB     pvalue 
## 0.15386752 0.01995935 0.28506413 0.02448254 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 140 
## - Total number of p<0.05 studies included into the analysis: k = 102 (72.86%) 
## - Total number of studies with p<0.025: k = 64 (45.71%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull   zHalf pHalf
## Right-skewness test     0.006 -8.767 0.000 -10.844     0
## Flatness test           0.036  2.090 0.982  12.905     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 48% (36.5%-59.6%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
##                Median power for detecting a SESOI of d = .20 
##                                                        0.204 
##                Median power for detecting a SESOI of d = .50 
##                                                        0.806 
##                Median power for detecting a SESOI of d = .70 
##                                                        0.977 
## Median power for detecting PET-PEESE estimate.PEESE estimate 
##                                                        0.183 
##               Median power for detecting 4/3PSM estimate.est 
##                                                        0.189

F-test of equality of variances

Mean vi for non-randomized designs

## [1] 0.04069351

Mean vi for randomized designs

## [1] 0.0660096

F-statistics

## [1] 0.6164787

F-test p-value

## [1] 0.1272321

Comparing effects based on student and non-student samples

## $`Model results`
## $`Model results`$test
##                    Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(studentSample)0    0.327 0.0456   7.16    <0.001  ***
## 2 factor(studentSample)1    0.453 0.0395  11.46    <0.001  ***
## 
## $`Model results`$CIs
##                     Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(studentSample)0    0.327 0.0456  Inf        0.237        0.416
## 2 factor(studentSample)1    0.453 0.0395  Inf        0.375        0.530
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  4.36      1       81 0.0398   *

Year of Publication

Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.

##                                    Estimate Std. Error       df   t value
## (Intercept)                      -0.1407496 0.09437155 85.97906 -1.491442
## scale(h5indexGSJournalMarch2016) -0.1697075 0.10151050 85.90369 -1.671823
## scale(publicationYear)           -0.2307599 0.08925779 85.99053 -2.585320
##                                    Pr(>|t|)
## (Intercept)                      0.13950560
## scale(h5indexGSJournalMarch2016) 0.09819827
## scale(publicationYear)           0.01141363

Comment: all the variables were centered for easier interpretation of model coefficients. See the negative beta for Publication Year. The more recent a publication, the lower the variance (better precision), controlling for H5.

Scatterplot year <-> precision

Size of the points indicate the H5 index (the bigger the higher) of the journal that the ES is published in.

## `geom_smooth()` using formula 'y ~ x'

Citations

Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.

##                                      Estimate Std. Error       df     t value
## (Intercept)                      -0.201059160 0.09182019 84.97688 -2.18970532
## scale(publicationYear)           -0.003047723 0.11144001 85.08921 -0.02734855
## scale(h5indexGSJournalMarch2016) -0.365198106 0.11472691 84.73581 -3.18319482
## scale(citationsGSMarch2016)       0.332488656 0.10530882 85.10522  3.15727251
##                                     Pr(>|t|)
## (Intercept)                      0.031286371
## scale(publicationYear)           0.978245772
## scale(h5indexGSJournalMarch2016) 0.002037484
## scale(citationsGSMarch2016)      0.002203503

Scatterplot precision <-> citations

The relationship between precision (sqrt of variance) and number of citations.

## `geom_smooth()` using formula 'y ~ x'

H5 index

Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.

##                                    Estimate Std. Error       df   t value
## (Intercept)                      -0.1141591  0.0968350 87.04277 -1.178903
## scale(h5indexGSJournalMarch2016) -0.1178194  0.1027219 86.98377 -1.146974
##                                   Pr(>|t|)
## (Intercept)                      0.2416493
## scale(h5indexGSJournalMarch2016) 0.2545376

Scatterplot precision <-> journal H5

The relationship between precision (sqrt of variance) and H5 index of the journal.

## `geom_smooth()` using formula 'y ~ x'

Decline effect

Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study.

##                           Estimate Std. Error       df  t value      Pr(>|t|)
## (Intercept)            0.001256607 0.08446073 86.44516 0.014878 0.98816379458
## scale(sqrt(vi))        0.361986905 0.08744592 79.13890 4.139552 0.00008628732
## scale(publicationYear) 0.114376928 0.08487961 97.53930 1.347519 0.18093479180

Citation bias

Do more highly-cited studies report larger effect sizes?

##                                     Estimate Std. Error       df    t value
## (Intercept)                       0.22906790 0.01409808 76.12334 16.2481604
## scale(publicationYear)           -0.01667034 0.01740063 82.82924 -0.9580309
## scale(h5indexGSJournalMarch2016) -0.04319160 0.01701388 64.19127 -2.5386092
## scale(citationsGSMarch2016)       0.02599613 0.01651275 85.24009  1.5743067
##                                                            Pr(>|t|)
## (Intercept)                      0.00000000000000000000000001855457
## scale(publicationYear)           0.34083527493986021106309181050165
## scale(h5indexGSJournalMarch2016) 0.01356484913606161198107447063421
## scale(citationsGSMarch2016)      0.11912106790969252678724643601527

P-curve for interaction effects

## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 52 
## - Total number of p<0.05 studies included into the analysis: k = 38 (73.08%) 
## - Total number of studies with p<0.025: k = 23 (44.23%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.128 -4.743 0.000 -5.482     0
## Flatness test           0.098  0.743 0.771  7.386     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 42% (23.3%-61.3%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no

P-curve and contour-enhanced funnel plots

Counts

Simple counts: 1. How often did authors test for moderation by attachment?

## 
##   0   1 
## 303  19
  1. Via validated tests: How often was tested for:
  1. Independence of awareness
## 
##   0 
## 323
  1. Lack of intention
## 
##   0 
## 320
  1. Efficiency of behavior
## 
##   0 
## 317
  1. Lack of control of behavior
## 
##   0 
## 321
  1. Via non-validated tests: How often was tested for:
  1. Independence of awareness
## 
##         0         1 
## 0.7080925 0.2167630
## [1] 51
  1. Lack of intention
## 
##   0 
## 323
  1. Efficiency of behavior
## 
##   0 
## 320
  1. Lack of control of behavior
## 
##   0 
## 322
  1. Whether researchers tested for innateness
## 
##   0 
## 320
  1. Skin temperature location
## 
## 1 2 
## 6 5
  1. Population type
## 
##         general special student 
##      28      84       8     226
  1. Control group type (1 = active controls)
## 
##  0  1 
##  6 37
  1. In how many countries the studies were conducted and what their mean distance was from the equator
##  [1] "USA"         ""            "China"       "Portugal"    "Singapore"  
##  [6] "Israel"      "Germany"     "South Korea" "Netherlands" "Japan"      
## [11] "Scotland"    "England"     "India"       "Canada"      "Switzerland"
## [16] "Poland"      "Italy"
  1. Number of independent studies
## [1] 84
  1. Number of papers
## [1] 33
  1. Lattitude
## Lattitude mean   Lattitude SD            Min            Max 
##       39.74729       10.84424        1.29686       57.16498

Further sensitivity analyses

Different step functions for the 3PSM

##    steps deltaModerate deltaSevere deltaExtreme
## 1 0.0025          1.00        1.00         1.00
## 2 0.0050          0.99        0.99         0.98
## 3 0.0125          0.97        0.97         0.95
## 4 0.0250          0.95        0.95         0.90
## 5 0.0500          0.80        0.65         0.50
## 6 0.1000          0.60        0.40         0.20
## 7 0.2500          0.50        0.25         0.10
## 8 0.5000          0.50        0.25         0.10
## 9 1.0000          0.50        0.25         0.10
## $deltaModerate
##     est      se  zvalue  pvalue    ciLB    ciUB       k   steps 
##   0.330   0.031  10.606   0.000   0.269   0.391 116.000   9.000 
## 
## $deltaSevere
##     est      se  zvalue  pvalue    ciLB    ciUB       k   steps 
##   0.240   0.037   6.402   0.000   0.166   0.313 115.000   9.000 
## 
## $deltaExtreme
##     est      se  zvalue  pvalue    ciLB    ciUB       k   steps 
##   0.133   0.032   4.165   0.000   0.070   0.196 115.000   9.000